A GUI for the Deaf-Blind: An additional concepts paper

People with some residual vision though still classified as blind may be able to read on screen patterns presented via a computer program whereby such a pattern can represent a textual character and a presentation of sequences of such patterns can represent words. These patterns can be adjusted to utilize the user’s remaining vision capacity. Previous research has shown that volunteers who where legally classified as blind could read simple sentences via such a system. This system is known as the Dynamic Pattern System (DPS) and is a test bed for research. Although this research involved a single pattern represented a single character the DPS system can also use a pattern to represent a word, or conversely, a sequence of patterns to represent a character. A modified DPS could become a ‘front-end’ to conventional IT applications software converting conventional text to and from DPS patterns. The original research into DPS was carried out using mainly blind and not deaf-blind volunteers yet DPS may have more application to the deaf-blind due to the wide availability of screen readers. Enhancing access to electronic media and communications for the blind and deaf-blind users has the potential to result in better quality of life outcomes and potentially enhance workforce participation. The DPS could possibly help, via enhanced equipment provision to increase equity of access for blind or blind-deaf to enable them to more readily access and interact with information. As such the DPS provides a possibility to transforming user’s lives via technology. This could assist in the retention of employees and to more fully align to antidiscrimination requirements. Introduction With much of today’s access of information coming via the Web or other electronic media there is a problem of access for those that are classified as blind and particular those that are deaf-blind. However, most people classified as blind have some residual sight. According to the Royal Blind Society of New South Wales 10% or less of the sight of a fully sighted individual is sufficient for a person to be legally classified as blind (RBS, 1996). There are many people classified as blind, but with some residual vision, that can perceive patterns on a computer screen, or on a computer projection screen, that may not be able to read even enlarged on-screen text. There are now readily available text to speech readers to read text that have become very common in modern home computer systems. However this will not be of use to many deaf (Leuterman, 2002) or deaf blind as they may not be able to comprehend the sounds from the reader . The Dynamic Pattern System (DPS) The DPS (Veal & Maj, 2001) has been developed that can replace text by an on screen pattern developed to fit the user’s remaining visual capacity (Veal, Maj, & Kohli, 2004). Up to five on screen coloured areas can form a pattern as shown in figure 1 and also the background colour may be set. Each pattern can represent a character and a sequence of characters can represent a word (Veal & Maj, 1998). The space between the pattern elements and the size and shape of these elements may also be adjusted. This is similar to the multiple uses of ships’ signalling flags (Maine Harbors, 2005). The delivery rate of these characters to the screen may be adjusted to suit the individual’s present preferences for reading via DPS. To avoid inducing fits in photo sensitive epileptics DPS does not allow presentation speeds of greater that five patterns per second well away from the ten patterns per second which might trigger the photo epileptic seizures (West Australian Newspapers, 1997). As well as a character being associated with a pattern the pattern representing the white space between words can be presented for a longer time period than other characters’ patterns. This is similar to Morse code where the inter-word space is greater than the inter-character space. Figure 1. The DPS main parameter setting screen for controlling the presentation of sequences of patterns to a volunteer. DPS also has the potential to be used to present simple diagrams whereby the direction of lines and turns are indicted by changing sets of patterns. (Veal & Maj, 2005). A modified DPS could become a ‘front-end’ to conventional IT applications software converting conventional text to and from patterns matched to the user’s remaining sight. Problems The need to undertake learning a whole new sequence of alpha-numeric characters can be demanding and would presumably only be thoroughly undertaken by someone who would not have another means of reading text (Seifert, 1997). To assist the learning process the DPS has drop down help files which are in the form of simple text files. This enables them to be read by the user as patterns by being fed into DPS. The help files can also be updated by the user or trainer to provide extra information should this be required as shown in Figure 2. Figure 2. The DPS help text can be also be read as dynamic patterns DPS can be designed so that for many people there is the opportunity to allow for redundant cues in patterns. Therefore should a user’s eye sight deteriorate then the same patterns could still be recognised via the remaining cues. Allowing for redundancy in patternsets may also allow more cues to be missed or misinterpreted during the reading process and yet the correct character to still be correctly identified. Many vision disabled suffer from congenital achromatopsia who have never been able to see colours and report similar effects when looking at shades of gray (Futterman, 2005). Two or more patterns may then be required to represent a single character. This is because with a maximum of five shapes per character two to the power five is only thirty two and the alphabet, numerical digits and punctuation marks required a greater number of representations. However, many users have non-congenital achromatopsia and can still see shades of gray as they may have once possessed colour vision (Futterman, 2005) and therefore a single pattern per character may suffice as the DPS can provide multiple shades of gray. Research previously undertaken at the Royal WA Institute for the Blind had demonstrated that a single set of patterns would not suit all users. Hence the DPS system has a screen to enable a researcher to change the background colour and the colour of the pattern elements associated with each character as shown in Figure 3. Figure 3 The DPS screen for selecting pattern element colours Multiple patterns representing a character A sequence of multiple patterns can also be used to represent a character. The sequence patterns shown below should be imagined as representing a letter. This use of the DPS may be regarded as a form of gesture communication (Segen & Kumar, 1998). For example gesture communication is also used by ground staff to direct aircraft on the ground and also by workers directing crane drivers. However this also slows down the delivery of the text. Figure 4 shows a sequence of patterns representing the given character for a particular patternset. At 0.0ms At 0.200ms At 0.400ms Figure 4. A sequence of patterns representing a textual character. Parallel character presentation There are some speed limitations due to the serial nature of DPS in that each character is sent in sequence. This does not allow users to read ahead and hence this slows down the reading process. However, there exits the possibility of simultaneously showing more than a single character on screen as shown below in Figure 5 which should be imagined as covering the whole of a screen. Figure 5. Parallel presentation of patterns representing two characters. Multiple simultaneous presentation of word objects A single DPS pattern can also be used to represent a word. Another possibility when using a computer projector is single on screen sequence of patterns to represent a word where this sequence of patterns being repeated continuously to represent that word. Multiple sequences of character presentation can also be presented on screen simultaneously. A sentence therefore could be seen as a line of such patterns shown on a screen. These patterns can be in line, as in the case of on screen sentences, but may be setup not to be inline to avoid a user’s blind areas. Some of the volunteers in previous tests exhibited such blind areas in the centre of their field of vision. Hence these sequences can be regarded as a word, a sentence, a paragraph or a whole document represented by on-screen DPS objects. The initial DPS experiments could be then understood to have been as a special case of these DPS objects that only showed a single object at a time using nearly the whole of the screen space. Limitations There is the problem of the time taken to set up even a limited character set. The automatic generation of patternsets based upon user perception could greatly assist this process (Gajos, Long, & Weld, 2006). To reduce the number patterns needed to be learnt the DPS treats upper and lower case letters as identical. Further work has been undertaken on the DPS to allow the background colour to vary as part of a characters representation and the effect of this change needs to be investigated. As an addition safety measure this group of the population was not used as volunteers. Another intended change is to allow individual pattern elements to vary size within a given pattern. The present version of DPS only allows a single size of pattern element for the whole patternset once this is chosen by the user. Varying the on screen position of pattern elements within a given pattern, rather than the whole patternset is also an intended change. Both of these changes will need to be investigated to discover how useful they are to the visually disabled. Social needs Enhancing access to electronic media and communications for deaf blind users has the potential to result in better quality of life outcomes (Kendrick, 2007), and potentially greater access to workforce and educational participation (Asakawa, 2005). For employer’s and Information Communication Technology (ICT) based companies. The DPS could possibly help, via enhanced equipment provision to increase equity of access, (Fewtrell, 1998) to enable retention of employees and to more fully align to antidiscrimination requirements (Schrage, 2000) such as the US Americans with Disability Act (ADA) (Ladner, 1989; Watenberg, 2004). ConclusionsProviding disabled users with tools to take part in the technologicaltransformation of modern society is a legal, ethical and economic imperative.Such access to and interaction with information could possible be enhanced byallowing the deaf-blind and blind who can still see large on screen patterns touse these patterns in place of conventional text. With the DPS acting as a“front-end” it could enable user’s to use conventional IT software and translateconventional text from and to the user’s on screen patterns. The additionalconcepts presented in this paper included a further emphasis on the deaf-blind,the use of now relatively cheap projectors which could allow the use of multiplecharacters to be delivered per screen to allow read ahead of characters muchas occurs in the processes of conventional reading. DPS could also possiblybenefit for some users if they could access a specially pre-designed characterset based upon their vision impairment classification. More research is neededto provide answers to these questions and to ascertain if the DPS can indeedtransform users’ lives via technology. ReferencesAsakawa, C. (2005, May 10). What's the web like if you can't see it? Paperpresented at the WWW2005, Chiba Japan.Fewtrell, B. (1998). Equitable access for the visually impaired., Incite, 10.Futterman, F. (2005, April 3